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Sample Estimates? 


Introduction 


Many users of Labour Force Survey (LFS) estimates are interested in the movement between 
successive monthly estimates. The current LFS design is very good for estimating this monthly 
movement. There have, however, been recent suggestions that estimates of movement could be 
improved by the use of so-called ‘matched sample’ estimates. This article considers the 
usefulness of matched sample estimates and describes the properties of such alternative 
estimates. It also makes recommendations on appropriate use of the LFS estimates. 


Why use the matched sample? 


The ‘matched sample’ between two months consists of individuals who were interviewed in the 
survey in both months. An estimate of movement based only on 

these individuals is called a matched sample estimate. This estimate is intuitively attractive 
compared to the movement based on the whole sample, which is affected by changes in the 
individuals sampled between the months. By using only the matched sample these changes are 
avoided, reducing the volatility of the estimates. However, the common sample has a reduced 
sample size, and this tends to increase the sampling error partially 

offsetting the reduced volatility on the matched sample estimates. 


Matched sample estimates are biased 


The LFS procedures aim to ensure that the individuals surveyed at each time point are a 
representative sample of individuals. The procedures do not ensure that the set of individuals 
interviewed in both of two successive months is a representative sample of the population at 
either time point. In practice, the matched sample will tend to omit individuals who are harder to 
contact or more mobile. Such people are more likely to have been missed or to have moved 
house between the surveys. They are also likely to have labour force characteristics that differ 
from the rest of the population. Thus matched sample estimates of movement will not represent 
these classes of people as adequately as the original estimates do. That is, they are open to 
‘bias’, in that they are likely to understate the movement in the whole population. 


The ABS does not produce matched sample estimates from the labour force survey. However, 
some matched sample data are published in ‘gross flows’ tables in Labour Force, Australia 
(6203.0). 


These tables are not aimed at producing estimates for the whole population, so they have not 
been specifically adjusted for that purpose. This means that they not only under-represent more 
mobile individuals, but also may have a different age and sex mix from that of the whole 
population. Matched sample estimates from this source would therefore be subject to additional 
variation or bias. 


Is a better estimate of movement available? 


The LFS sample is actually divided into eight groups, known as rotation groups. Each rotation 
group is a representative sample of the Australian population in 

its own right. Estimates from these rotation groups can be used to produce alternative labour 
force estimates that are not biased in the ways just described. 


Between any pair of successive months, the sample in one of the rotation groups is changed. 
The other seven rotation groups contain the same sample of 

dwellings (though they will include some different individuals due to mobility of the population). 
An estimate of movement could be produced based on the seven ‘matched rotation groups’. 
(These rotation groups contain individuals other than those in the matched sample. This allows 
them to be representative of the population, though with slightly lower month-to-month 
correlations than the matched sample alone.) 


Calculations show that matched rotation group estimates of movement of employed persons 
would have standard error about 10% lower than that of a movement based on the published 
estimates. (Standard error is a measure of the sampling error on an estimate-for details on 
standard error in the LFS see Labour Force Australia (6203.0)). The improvement for estimates 
of unemployed persons would be smaller (under 2%) because they have lower correlation 
between months. 


Why doesn’t the ABS use matched rotation group estimates of movement? 


The ABS could produce valid estimates of movement using the matched rotation group 
approach. However, publishing these estimates would raise some difficult issues. First, many 
users would prefer the estimate of movement to equal the difference between the published 
monthly estimates of level. To achieve this would require changing the monthly estimates, so that 
each monthly estimate of level is obtained by adding the estimate of movement to the level 
estimate from the previous month. However, the monthly estimates of level produced by this 
method would have higher sampling error than the current ones, with sampling error increasing 
over time. These estimates could stray significantly over time from the best available estimates of 
the level of the series. 


Second, the ABS does not encourage users to base their decisions on the individual month-to- 
month movements. Decisions about the direction of labour force series are better made by 
examining the trend series. Month-to-month movements often go in a different direction from the 
underlying direction of the time series, and so can be misleading. Given the importance of the 
level and trend series, it would be poor practice to base the series on an estimate optimised for 
monthly movement at the expense of the trend estimate. 


(As a side issue, some users prefer to look at quarterly averages in order to obtain a broad 
picture of the direction of the economy. While this is valid, the trend estimates published by the 
ABS tend to detect and confirm turning points in a monthly series much earlier than an 
examination of the quarterly averages.) 


Third, the cost and confusion that could be caused by a change to ABS estimation practices 
would offset the relatively small improvements that apply to movement estimates. 


Finally, the matched rotation group estimates of movement are just one example of a whole class 
of estimates known as composite estimates. If a composite estimate is used, there are more 
sophisticated approaches that yield greater gain. 


Composite estimates 


While matched sample estimators give all the weight to the matched sample, and none to the 
sample dwellings which change between months, composite 

estimators give most weight to the common dwellings, but also put some weight on the changed 
dwellings. Composite estimators have been derived that achieve improvements for monthly 
estimates of level, movement and trend (for employed persons the improvements in standard 
error are about 6%, 12% and 6%, respectively). 


These composite estimators achieve consistency between the different estimates by revising the 
last few monthly estimates when new data arrives. This takes advantage of the correlation of 
successive monthly estimates to improve earlier estimates when new information arrives. While 
this approach is reasonable, it is possible that regular revisions would be unacceptable to users. 
The need to gain user acceptance of revisions is a factor against using this form of composite 
estimation. 


A comparison of published, matched rotation group, and composite estimates 


Matched rotation group estimates and composite estimates were produced for comparison with 
published figures for experimental purposes. The variable studied was the proportion of the 
population employed. Proportion employed is used because it has a higher correlation month to 
month than proportion unemployed, and hence greater scope for improving the movement 
estimates. The matched rotation group estimates are based on adding successive estimates of 
monthly movement to an initial published figure from November 1990. Graph 1 shows the 
resulting matched rotation group series alongside the original series and the composite estimate 
series. 


The graph suggests a small systematic difference between the non matched rotation groups and 
the remaining groups, with the difference accumulating over time, in addition to any difference 
arising from sampling error. Sensitivity to such a systematic difference is an unfortunate property 
of the matched rotation group estimates. The composite estimates do not have this problem. 


To use the matched rotation group estimates over a long period would require at least occasional 
revision of the series to bring it into line with the best estimate 

of level. One approach is to calculate the average monthly discrepancy over a long period, and 
add it in to each matched rotation group movement estimate. 

This is equivalent to adjusting each matched rotation group movement by the same amount so 
that they add to the long term movement of the original series. 


This ‘adjusted matched rotation group’ series is closer to the original series, but still strays from it 
for extended periods. This is clear from Graph 2, which 

gives the difference from the original series of both the adjusted matched rotation group 
estimates and the composite estimates. It can be seen that the 

adjusted matched rotation group series is above the original series for the period around late 
1990 to early 1993, and then below for a period from early 1993 to 

late 1995. The differences are too large to be explained by the sampling error on the original 
series, given the number of successive time points for which 

the discrepancy was large. This suggests that a single adjustment for the whole series was 
insufficient, and more regular adjustments (and hence revisions) would be needed to give a good 
series. 


In contrast, the composite estimator does not stray far from the original series for long. It is 
interesting to note that the composite estimator often moves in the same way as the matched 
rotation group estimator, but not as far, or for as long. In a sense, the composite estimator is a 
compromise between the two 

estimates, having short-term properties like the matched rotation group estimator and long-term 
properties more like the original series. 


GRAPH 1. COMPARISON OF ESTIMATES, PROPORTION EMPLOYED 
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GRAPH 2. DIFFERENCE FROM ORIGINAL ESTIMATE, PROPORTION EMPLOYED 
(percentage points) 
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GRAPH 3. COMPARISON OF ESTIMATES, PROPORTION EMPLOYED 
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Note, however, that the graphs give the composite estimator at its final value after revisions - 
they do not illustrate the sequence of small revisions which each 


composite estimate undergoes in the first few months after it appears. The total of these 
revisions was measured for each month in the series - the average 

magnitude of these total revisions (ignoring whether they are positive or negative) was 0.07 
percentage points, with the largest total revision being 0.25 

percentage points. For comparison, the average month to month movement (ignoring whether 
they are positive or negative) over the same period was 0.5 

percentage points. 


Measuring longer term movements 


The matched rotation group estimates are somewhat better than the original estimates as 
indicators of month-to-month movement. This remains true for adding a small number of these 
movements together to obtain a longer term movement, for periods up to about six months. The 
composite estimator is better still, for both short and longer term movements. 


For measuring movements over more than six months, the matched rotation group estimates 
quickly deteriorate compared to the original estimates. For movement between months a year 
apart the matched rotation group estimator has 16% greater standard error than the original. 
(This compares poorly with the 

composite estimator, for which yearly movement has a 14% lower standard error than the 
original.) 


Graph 3 shows a particularly bad period (in 1993) where adding successive matched rotation 
group movements to the January 1993 figure would significantly understate the movement over a 
twelve month period. This occurs because over this period the newly selected dwellings tended 
to have higher employment than those they replaced. These upward influences were completely 
discounted by the matched estimates. The composite estimates make better use of the data, and 
take values similar to the original for this period. 


Conclusion 


Estimates of movement based on matched samples of persons can achieve lower sampling error 
than the published estimates, but are open to significant bias. 

As aresult, the ABS does not recommend use of estimates based only on the matched sample 
(as published in ‘gross flows’ tables). This article described a less biased estimate based on 
matched rotation groups, and discussed its properties. A still better method known as composite 
estimation was also discussed. 


All the alternative estimators discussed have drawbacks compared to the current published 
estimates. In particular, if an estimate that does not drift from the best level estimate is sought, 
revision is required. For this reason the ABS official estimates will continue to be based on the 
current approach. Composite estimates show some promise, and the ABS will investigate these 
further. Users interested in the direction of the labour force series are advised to use the 
published trend series rather than placing too much emphasis on the month-to-month 
movements. 


This feature article was contributed by Phillip Bell, Labour Force, ABS. 
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